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Abstract 

This digest reports on a descriptive study examining two questions related to adult English language learners’ educational 
level gains in the National Reporting System for Adult Education (NRS), as measured by the standardized oral proficiency 
assessment BEST Plus: (1) What is the relationship between instructional hours and educational level gain on BEST Plus ? 
and (2) What is the relationship between intensity of instruction and educational level gain on BEST Plus ? To provide 
guidance to users on the number of instructional hours needed for students to show a level gain on BEST Plus, the Cen¬ 
ter for Applied Linguistics collected pretest and posttest data on more than 6,500 students from two states. The largest 
number of students pretested with BEST Plus tested into the NRS Beginning ESL Literacy level (49%). The fewest were 
placed in the NRS Advanced ESL level (7%). 

Statistical analyses were conducted to see how examinee performance at each level varied according to number of 
instructional hours and intensity of instruction. Instructional hours were defined as the number of hours that students actu¬ 
ally attended class, ranging from fewer than 60 hours to more than 140 hours. Intensity of instruction was defined as how 
often students attended class over a given period of time, ranging from low intensity (e.g., 100 hours of instruction over 
250 days) to high intensity (e.g., 100 hours of instruction over 75 days). Results showed that across NRS educational func¬ 
tioning levels, the greater the number of instructional hours, the higher the percentage of students who made level gain. 
There was also a general trend toward greater NRS level gain for students with high levels of instructional intensity than 
for those with low intensity. Intensity of instruction had the greatest impact on students at the Beginning ESL Literacy, 
Low Intermediate, and Advanced ESL levels. Additional research is needed to better understand differences in level gain 
and the learner and programmatic factors, including number of instructional hours and intensity of instruction, that may 
influence oral language proficiency as measured by NRS level gain. 


Defining Adult English Language 
Learners' Educational Level Cain 

The National Reporting System for Adult Education (NRS) 
is the accountability system for federally funded adult ba¬ 
sic education, adult secondary education, and English as a 
second language (ESL) education in the United States. (See 
www.nrsweb.org for more information.) Upon enrollment, 
students in these programs place into one of six educational 
functioning levels based on their pretest scores. Their prog¬ 
ress through these levels is reported each year by state de¬ 
partments of education to the U.S. Department of Educa¬ 
tion, Office of Vocational and Adult Education. Each state 
negotiates a target percentage of students at each educational 
functioning level that will advance at least one level (educa¬ 


tional level gain) each year. Educational level gain in lan¬ 
guage and literacy is measured by pretesting students with 
an approved standardized assessment, then posttesting them 
with an equivalent form of the same assessment after a pre¬ 
determined number of instructional hours or at the end of an 
instructional cycle. The minimum number of instructional 
hours recommended between pretesting and posttesting for 
NRS-approved assessments ranges from 40 to 120 hours 
(National Reporting System, 2006). 

Adults learning English as a second language are placed 
in NRS ESL levels that range from Beginning ESL Literacy 
to Advanced ESL. The primary objective of many of these 
learners is to develop the English language and literacy skills 
they need to meet their personal, community, academic, and 
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employment goals (National Center for ESL Literacy Educa¬ 
tion, 2003). Adult ESL programs focus on helping students 
develop the oral communication and literacy skills they need 
to attain their goals, which may include helping their chil¬ 
dren in school, obtaining a high school diploma, entering a 
postsecondary education program, getting a better job, and 
earning U.S. citizenship. 

Learner and Program Factors Related to the 
Rate of English Acquisition by Adult English 
Language Learners 

In 2003-2004 (the year for which the most recent data are 
available on students in federally funded programs), 1,172,579 
students were enrolled in federally funded adult ESL classes; 
36% of these students attained an educational level gain after 
a course of instruction (U.S. Department of Education, Office 
of Vocational and Adult Education, 2006). Given the variety 
of instructional, environmental, and personal factors that af¬ 
fect second language acquisition, it is difficult to say what 
factors might have the greatest impact on how quickly adult 
ESL students make gains in learning English. Adult learners 
come to ESL programs with a variety of prior educational 
and life experiences, English language proficiency levels, and 
educational goals. Learners also differ in their opportunities 
for language acquisition outside the classroom and in the cir¬ 
cumstances that affect their participation in class. Some may 
work in jobs that require them to use English, whereas others 
may not. Some learners are able to attend ESL classes several 
times a week and have opportunities to study English outside 


the program, while others do not. Other factors such as lan¬ 
guage aptitude, age, and motivation may also play a part in 
rates of English language acquisition. 

Specific program factors may also affect learners’ educa¬ 
tional gains. Adult education programs are often tailored to 
take advantage of the few hours (typically 4 to 8 hours per 
week) that adult learners are available to study. Instruction 
may focus on a limited number of learner goals (e.g., finding 
a better job or helping children with their homework) or may 
span a wide variety of content topics and language skills. It is 
important to keep all of these factors in mind when consider¬ 
ing expectations for adult ESL educational gain and attain¬ 
ment of goals. See Table 1 for an overview of these factors. 
Moss and Ross-Feldman (2003) offer information about how 
to approach English language acquisition in adult students 
through an understanding of what the research says about 
learner motivation (e.g., Dornyei, 2002), interaction (e.g.. 
Long, 1996), task-based learning (e.g., Ellis, 2000), and vo¬ 
cabulary (e.g., Coady & Huckin, 1997). 

BEST Plus Educational Level Cain Study 

BEST Plus is one of the standardized assessments approved 
for accountability reporting for the NRS. BEST Plus assesses 
the oral proficiency of adult English language learners across 
three components: 

• Listening comprehension measures how well students 
understand spoken English. 

• Language complexity measures the depth and breadth 
of students’ use of conversational English. 


Table 1. Learner and Program Factors Affecting Adult English Language Acquisition and Instruction 
Learner factors Program factors 


• Time available for instruction 

• Opportunities to use English outside the classroom 

• Age 

• Language and cultural background 

• Level of prior education 

• Degree of first language literacy 

• Personal conditions (e.g., visual or auditory 
impairment) 

• Personal motivation 

• Other demands of life (e.g., work, family) 


• Instructional setting (e.g., school, work site) 

• Type of instruction (e.g., classroom, tutoring) 

• Type of entry (open vs. managed enrollment) 

• Time of instruction (day, evening, weekend) 

• Length of instructional program 

• Level of resources 
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• Communication measures how well students can be 
understood when speaking English. 

Following a complete test administration, a score report is 
generated that expresses the final test results in terms of a BEST 
Plus scale score and a Student Performance Level (SPL). (See 
www.cal.org/caela/esl_resources/slspls.html for SPL descrip¬ 
tors.) BEST Plus scale scores are also aligned with the NRS 
ESL educational functioning levels (see Table 2). 

Since the launch of BEST Plus in 2003, many users have 
expressed interest in knowing the number of instructional 
hours needed for learners to show a level gain on this assess¬ 
ment. To provide some guidance to users on this issue, the 
Center for Applied Linguistics collected pretest and posttest 
data from two states and conducted statistical analyses to see 
how examinee performance at each of the NRS educational 
functioning levels varied according to number of instruction¬ 
al hours and intensity of instruction. Two research questions 
were addressed: 

Question 1: What is the relationship between instructional 
hours and educational level gain on BEST Plus ? 

Question 2: What is the relationship between intensity of 
instruction and educational level gain on BEST Plusl 


Table 2. Alignment of NRS Levels, Student 

Performance Levels, and BEST Plus Scale Scores 


NRS Level 

SPL 

BEST Plus 

Score Range 

Beginning ESL Literacy 

0-1 

400 and below 

Low Beginning ESL 

2 

401 to 417 

High Beginning ESL 

3 

418 to 438 

Low Intermediate ESL 

4 

439 to 472 

High Intermediate ESL 

5 

473 to 506 

Advanced ESL 

6 

507 to 540 

Exit from NRS ESL levels 

7 

541 and higher 


Background Information 

BEST Plus pretest and posttest scores (for tests adminis¬ 
tered between April 2004 and June 2005) were collected 
for 7,208 adult ESL students in Massachusetts and Illinois. 
Program staff also contributed attendance data for the same 


students during this time period. BEST Plus has been used 
in Illinois since 2003 (approximately 56,000 tests had been 
administered as of mid-2007) and in Massachusetts since 
2004 (approximately 86,000 tests had been administered 
as of mid-2007). A total of 6,599 examinees with complete 
data for analysis remained in the data set. The pretest NRS 
educational functioning levels of these examinees are shown 
in Table 3. (Percentages in this report are rounded up to the 
nearest whole number, so totals may be slightly higher or 
lower than 100%.) 


Table 3. Distribution of Participants in NRS Levels Based 
on BEST Plus Pretest 


Initial NRS Level 

Number 

Percentage 

Beginning ESL Literacy 

3,209 

49% 

Low Beginning ESL 

519 

8% 

High Beginning ESL 

709 

11% 

Low Intermediate ESL 

989 

15% 

High Intermediate ESL 

718 

11% 

Advanced ESL 

455 

7% 

Total 

6,599 

100% 


At the time that these data were collected, the original 
(1999-2005) NRS ESL educational functioning level tables 
were in use, comprising Beginning ESL Literacy, Beginning 
ESL, Low Intermediate ESL, High Intermediate ESL, Low 
Advanced ESL, and High Advanced ESL. However, these 
data were analyzed using the current NRS ESL levels (re¬ 
vised as of July 2006). For example, the old Beginning ESL 
was split into Low Beginning ESL and High Beginning ESL 
and the old High Advanced ESL was removed. Under the re¬ 
vised NRS level tables, the largest number of students pre¬ 
tested using BEST Plus were placed in the NRS Beginning 
ESL Literacy level (49%). The fewest number of students 
were placed in the NRS Advanced ESL level (7%). These 
numbers are similar to the overall breakdown of levels of all 
students enrolled in federally funded adult ESL programs 
in the United States in 2003-2004, the latest year for which 
these data are available (U.S. Department of Education, Of¬ 
fice of Vocational and Adult Education, 2006). 


3 



Table 4 shows the number and percentage of students in 
the study who made level gain. The greatest rate of level gain, 
regardless of number of instructional hours, occurred in Low 
Beginning (78%) and High Beginning (77%) ESL levels, with 
Beginning ESL Literacy (54%) and Advanced ESL (55%) 
making the lowest rates of level gain. Overall, 60% of the stu¬ 
dents made one or more level gains. 


Table 4. Number of Students Attaining NRS Educational 
Level Gain by NRS Level 


Initial NRS Level 

Number 
making 
level gain 

Percentage 
making 
level gain 

Beginning ESL Literacy 
n=3,209 

1,720 

54% 

Low Beginning ESL 

n=519 

407 

78% 

High Beginning ESL 

n=709 

543 

77% 

Low Intermediate ESL 

n=989 

614 

62% 

High Intermediate ESL 

n=718 

408 

57% 

Advanced ESL 

n=455 

252 

55% 

Total 

n=6,599 

3,944 

60% 


Question 1: What Is the Relationship Between 
Instructional Hours and Educational Level Cain 
on BEST Plus? 

BEST Plus was administered as a posttest to the 6,599 stu¬ 
dents after they received instruction ranging from 2 to 512 
instructional hours. Approximately 38% of students in the 
140 or more hours group received more than 200 instruc¬ 
tional hours. For these data sets, instructional hours is de¬ 
fined as the number of hours that students actually attended 
class, not the total number of hours offered in the program. 


The percentages of all examinees (regardless of NRS edu¬ 
cational functioning level) who attained at least one level 
gain between the pretest and posttest are delineated by num¬ 
ber of instructional hours in Table 5. Students with 140 or 
more instructional hours had the highest percentage of edu¬ 
cational level gain on BEST Plus (70%), and students with 
fewer than 60 instructional hours had the lowest percentage 
of level gain (53%). 


Table 5. Number of Students Attaining NRS 

Educational Level Gain by Instructional Hours 


Instructional Hours 

Attained Level Gain 

Below 60 Hours 

n=l,101 

n=2,066 

53% 

60 to 79 Hours 

n=660 

n=l,194 

55% 

80 to 99 Hours 

n=611 

n=l,015 

60% 

100 to 119 Hours 

n=391 

n=630 

62% 

120 to 139 Hours 

n=331 

n=482 

69% 

140 or More Hours 

n=850 

n=l,212 

70% 

Total 

n=3,944 

n=6,599 

60% 


Table 6 breaks down the rates of level gain according to 
NRS educational functioning level and instructional hours. 
There is a general trend toward greater level gain with more 
instructional hours across all NRS levels. For each of the NRS 
educational functioning levels examined, a greater percent¬ 
age of students who received 140 or more instructional hours 
made level gain compared to those students who received 
fewer than 60 instructional hours. However, the percentage 
of level gain is not always consistent with increasing numbers 
of instructional hours. These inconsistencies cannot be ex¬ 
plained given the purely descriptive data that were analyzed. 
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Table 6. NRS Level Cain Related to Instructional Hours by NRS ESL Educational Functioning Level 


Instructional 

Hours 

Beginning 

ESL 

Literacy 

n=l,720 

Low 

Beginning 

ESL 

n=407 

High 

Beginning 

ESL 

n=543 

Low 

Intermediate 

ESL 

n=614 

High 

Intermediate 

ESL 

n=408 

Advanced 

ESL 

n=252 

Below 60 

n=536 

n=l 12 

n=160 

n=146 

n=96 

n=51 

Hours 

46% 

75% 

72% 

60% 

54% 

50% 

60 to 79 Hours 

n=284 

n=86 

n=85 

n=93 

n=68 

n=44 


52% 

79% 

69% 

51% 

49% 

48% 

80 to 99 Hours 

n=247 

n=57 

n=95 

n=105 

n=74 

n=33 


56% 

72% 

79% 

60% 

58% 

45% 

100 to 119 

n=159 

n=34 

n=52 

n=75 

n=42 

n=29 

Hours 

54% 

81% 

81% 

68% 

58% 

64% 

120 to 139 

n=120 

n=37 

n=52 

n=49 

n=43 

n=30 

Hours 

62% 

86% 

87% 

66% 

64% 

67% 

140 or More 

n=374 

n=81 

n=99 

n=146 

n=85 

n=65 

Hours 

67% 

84% 

83% 

71% 

62% 

66% 


Summary of Results: Instructional Hours 

Across NRS educational functioning levels, the general trend 
( with a few exceptions at some levels) is that the greater the 
number of instructional hours, the higher the percentage of 
students who made level gain. This is particularly true of 
students who pretested at the Beginning ESL Literacy level 
(21% difference between the fewest number and the greatest 
number of instructional hours) and the Advanced ESL level 
(16% difference). A high percentage of students in the Low 
Beginning and High Beginning levels made level gain, re¬ 
gardless of the number of instructional hours. 

Question 2: What Is the Relationship Between 
Intensity of Instruction and Educational Level 
Cain on BEST Plus ? 

Student pretest and posttest performances on BEST Phis 
were also compared to intensity of instruction (i.e., how of¬ 
ten students attend class). Measuring intensity of instruction 
provides different information about student attendance than 
measuring instructional hours. For this study, three levels of 
intensity were defined, based on a ratio of total hours of in¬ 
struction divided by days between pretest and posttest: 

• Low intensity (ratio less then .50); for example, 100 
hours of instruction over 250 days 


• Mid intensity (ratio between .50 and .99); for example, 
100 hours of instruction over 150 days 

• High intensity (ratio of 1.00 or above); for example, 100 
hours of instruction over 75 days 

Table 7 illustrates a general trend toward greater rates of NRS 
level gain for students with high intensity instruction (66% 
attained level gain) than for those with low intensity instruc¬ 
tion (56% attained level gain). 


Table 7. Overall NRS Level Gain by Intensity 
of Instruction 


Intensity Level 

Attained Level Gain 

Low intensity 

n=l,142 

n=2,049 

56% 

Mid intensity 

n=2,265 

n=3,733 

61% 

High intensity 

n=537 

n=817 

66% 

Total 

n=3,944 

n=6,599 

60% 
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Table 8 shows the rates of level gain according to NRS 
levels and intensity of instruction. Although all NRS levels 
demonstrated increased level gain with increased intensity 
of instruction, Table 8 indicates that intensity of instruction 
had the greatest impact on students at the Advanced ESL 
level (19% difference in rates of level gain between low in¬ 
tensity and high intensity groups). 

Summary of Results: Intensity of Instruction 

Overall, there was a tendency for a higher percentage of stu¬ 
dents in more intensive instruction to achieve level gain: 66% 
of students in high intensity instruction made a level gain com¬ 
pared with 56% of students in low intensity instruction. This 
tendency is strongest in the Advanced ESL, Low Intermedi¬ 
ate, and Beginning ESL Literacy levels, where the differences 
in level gain between low and high intensity range from 10% 
to 19%, as opposed to 4% to 8% for the other levels. 

Conclusion 

Overall, NRS level gain was achieved for 60% of the 6,599 
students whose pretest and posttest scores were included in 
the analyses. Of particular note, 78% of the Low Beginning 
ESL students and 77% of the High Beginning ESL students 
in the study achieved NRS level gain regardless of the number 
of instructional hours. These rates contrast with lower rates 
for students at the other NRS levels. They are also higher 
than the overall average of 36% level gain reported for all 
students in federally funded programs in 2003-2004 (U.S. 
Department of Education, Office of Vocational and Adult 


Education, 2006). Intensity of instruction also affected NRS 
level gain, with the greatest impact found in students at the 
Advanced ESL level. 

It is not possible to pinpoint the reasons for the differ¬ 
ences among levels based solely on the data analyzed in 
this study. The differences may indicate that students are 
acquiring more oral English language at a faster rate at Low 
Beginning ESL and High Beginning ESL levels than at oth¬ 
er levels. This increased rate of acquisition may be related 
to a variety of factors, such as the curriculum and materials 
used at these levels, or beginning students’ levels of motiva¬ 
tion. In accordance with lower level students’ limited range 
of knowledge and skills in English, beginning ESL instruc¬ 
tion usually focuses on a much more limited set of infor¬ 
mation (e.g., personal information, numbers, household 
vocabulary, simple sentences). As adult English language 
learners’ oral proficiency develops, a much greater range 
of skills must be demonstrated (e.g., supporting opinions, 
comparing and contrasting, elaborating on experiences) 
that may require a proportionally greater amount of time in 
order to show progress. 

Second language acquisition theories may also be consid¬ 
ered. (See Doughty & Long, 2005, for an overview of related 
hypotheses.) For example, the silent period hypothesis pro¬ 
poses that beginning language learners go through a period 
in which their receptive skills are stronger than their produc¬ 
tive skills. Another theory involves language fossilization, in 
which language learners reach a plateau, typically at more 


Table 8. NRS Level Cain by Intensity of Instruction 


Intensity Level 

Beginning 

ESL 

Literacy 

n=l,720 

Low 

Beginning 

ESL 

n=407 

High 

Beginning 

ESL 

n=543 

Low 

Intermediate 

ESL 

n=614 

High 

Intermediate 

ESL 

n=408 

Advanced 

ESL 

n=252 

Low 

n=548 

n=116 

n=165 

n=140 

n=121 

n=52 

Intensity 

50% 

77% 

75% 

55% 

57% 

49% 

(<0.50) 







Mid 

n=935 

n=247 

n=321 

n=386 

n=236 

n=140 

Intensity 

55% 

78% 

77% 

64% 

56% 

54% 

(0.50 - 0.99) 







High 

n=237 

n=44 

n=57 

n=88 

n=51 

n=60 

Intensity 

60% 

83% 

83% 

68% 

61% 

68% 

(>1.00) 
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advanced stages of their language acquisition, and have dif¬ 
ficulty making further progress. 

Additional research is needed to identify (1) reasons 
for the differences in level gain found in this study and (2) 
learner and programmatic factors, beyond number of instruc¬ 
tional hours and intensity of instruction, that may influence 
NRS level gain. The results of the statistical analyses show 
that the number of instructional hours had a greater effect on 
overall group NRS level gain, as measured by BEST Plus, 
than instructional intensity. It should be noted that this study 
describes clean data related to 6,599 participants. However, 
these participants were not randomly selected; personal, 
environmental, and programmatic variables were not con¬ 
trolled; and there were not equal numbers of students from 
each NRS educational functioning level. Because this report 
documents a descriptive study rather than a controlled cor¬ 
relational study, these preliminary results cannot be inter¬ 
preted as cause-and-effect relationships among the variables. 
Rather, these preliminary results can serve as the basis for 
examining the NRS level gain of the 6,599 adult ESL stu¬ 
dents whose data were included in the analyses. In order to 
make stronger correlational claims between instructional 
hours and NRS level gain as measured by BEST Plus, more 
complete data about randomly selected examinees and their 
programs need to be collected and analyzed in a more ex¬ 
perimental study. 


Recommendations for Adult ESL Programs 

The Center for Applied Linguistics continues to recom¬ 
mend 80 to 100 instructional hours between pretesting 
and posttesting on BEST Plus. The findings from this 
descriptive study regarding instructional hours support 
and strengthen this recommendation. This should be 
taken into consideration, along with other factors af¬ 
fecting second language acquisition and adult English 
language learners, such as those described above, when 
determining state and local assessment policy for pre- 
and posttesting. 

To increase the likelihood of NRS level gain by adult 
English language learners, adult ESL programs should 
consider doing the following: 

• Conduct a needs assessment of local learners to de¬ 
termine the best days, times, and locations to meet 
the scheduling needs of the adult ESL population in 
the community. 

• Offer a variety of available instructional hours at vari¬ 
ous sites based on the needs assessment. Identify ways 
to increase student attendance and persistence specific 
to the local area. 

• Provide information to students, instructors, 
funders, and community stakeholders about the 
number of instructional hours needed to show edu¬ 
cational level gain. 

• Inform new students at orientation and during intake 
processes of the importance of class attendance and 
participation in pretesting and posttesting; make sure 
students understand how pretesting and posttesting 
with a standardized assessment relate to the program’s 
funding sources and ability to track student progress. 

• Train instructors to make the most of instructional 
time by conducting student needs assessments, de¬ 
veloping effective lesson plans, selecting appropriate 
materials, and evaluating students’ progress on an on¬ 
going basis. 

• Teach students how to apply and reinforce the English 
language classroom instruction they receive to their 
interactions outside of the classroom, as doing so will 
enhance their English language development. 
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